Electronic watermark detection apparatus and method

ABSTRACT

Electronic watermarks in digital documents can be detected, even if the content of the digital documents has been subjected to distortion or other attempts to hide or destroy the watermarks. One method involves inputting the distorted image and comparison information, the comparison information including at least one of the original image or information used for embedding the electronic watermark; dividing a domain of the original image into a plurality of patches, based on the comparison information; inputting affine parameters of a predetermined patch from among the patches in the original image; extracting a patch candidate from the distorted image; using a predetermined electronic watermark detection method, judging whether the patch candidate in the distorted image adequately correlates with a neighboring patch in the original image; when the judging indicates an adequate correlation, outputting a part of the electronic watermark. Another method involves determining a correspondence between patches in an original image having n patches, and patches in a distorted image; more specifically, the method involves inputting predetermining affine parameters of one of the plurality of predetermined patches; setting the input predetermining affine parameters of the one of the plurality of patch as initial affine parameters for investigating a new patch; and investigating a correspondence of the new patch of the distorted image and the new patch of the original image by slightly modifying the set input predetermining affine parameters.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority under35 U.S.C. § 119 to Japanese Patent Application No. 2000-192358, filedJune 27, 2000, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to electronic watermark detectionapparatus and methods. More specifically, the invention relates toapparatus and methods for detecting electronic watermarks in digitaldocuments, even if the content of the digital documents has beensubjected to distortion or other attempts to hide or destroy thewatermarks.

2. Discussion of the Background

In recent years, electronic digital content, such as sound, music,active movies, and still pictures, have come to circulate widely. Muchdigital content is protected under copyright. In order to protect thecopyright of the digital content, various electronic watermark methodsare used.

An electronic watermark method comprises embedding an electronicwatermark within the digital content so that detecting and extractingthe digital watermark is difficult. According to the intended use, theelectronic watermark indicates identification information of a copyrightperson or a user, rights information of a copyright owner, useconditions, secret information required at the time of use, copy controlinformation, and so on.

The digital content in which the electronic watermark is embedded isoften geometrically distorted by various normal operations of a user, orby intentional attack. As an example of user's normal operations, thereis a change of the display size of an image. As an example ofintentional attack of the user, there is a distortion of an image.

With regard to the electronic watermark method, it is required that theelectronic watermark does not disappear, is not altered, and is able tobe extracted, even if geometrical distortion is provided to the digitalcontent. The requirement is called robustness.

Generally, geometrical distortion is classified into global distortionand local distortion.

Global distortion is distortion of a whole image, and may be caused byscaling, rotating, and/or parallel displacing. Global distortion can beexpressed as an affine transformation formula. FIG. 1 illustrates how anoriginal image I becomes a distorted image I′ by global distortion. Apoint a(x, y) of the original image I is displaced to a point a′(x′, y′)of the distorted image I′ by global distortion. In this case, scalingand rotating can be expressed with four parameters, each being acomponent a11, a12, a21, and a22 of a 2×2 matrix. Parallel displacingcan be expressed with two parameters (b1, b2). Therefore, globaldistortion can be expressed with the following formula with these sixparameters a11, a12, a21, a22, b1, and b2. $\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime}\end{pmatrix} = {{\begin{pmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{pmatrix}\begin{pmatrix}x \\y\end{pmatrix}} + \begin{pmatrix}b_{1} \\b_{2}\end{pmatrix}}} & (1)\end{matrix}$

On the other hand, local distortion means distortion in which each pixelof the whole original image is displaced by a different 2-dimensionalvector. FIG. 2 shows that an original image I becomes a distorted imageI′ by local distortion. Therefore, local distortion can be expressedwith the following formula as 2-dimensional generalized-coordinateconversion. $\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime}\end{pmatrix} = \begin{pmatrix}{f\left( {x,y} \right)} \\{g\left( {x,y} \right)}\end{pmatrix}} & (2)\end{matrix}$where functions f and g are arbitrary functions. Although geometricaldistortion may also include clipping, for brevity its description is notspecifically included here.

The position of each pixel of the distorted image provided bygeometrical distortion displaces the position of each pixel of anoriginal image. An example of a detection method of electronicwatermarks with consideration to displacing position of a pixel is shownin U.S. Pat. No. 6,108,434 (Cox et al.).

The Cox et al. method finds a best corresponding shifting position byrepeatedly comparing with a predetermined block of an original image,while shifting a position of a block of a distorted image little bylittle on the basis of the position of the predetermined block of theoriginal image. If the distorted image is shifted based on the bestcorresponding shifting position, the distorted image is concluded tocorrespond to the original image. Thereby, the shifting position of thedistorted image corresponding to a original image can be compensated,and the electronic watermark in the distorted image can then bedetected.

The Cox et al. method involves a large computational load for arithmeticcalculations, since it is necessary to search for a large number ofpossible shifting positions. With regard to global distortion, as justto find the shifting position of a single arbitrary block, the Cox etal. method is considered to be an effective resolution method.

The geometrical distortion that a user performs on an image is localdistortion in many cases. In practice, it is difficult to detect theshifting position of each pixel by local distortion.

In local distortion, since vectors of neighboring pixels are almost thesame, neighboring pixels can be treated as a small block unit. Even iflocal distortion is treated as a small block unit, it is difficult inpractice since the number of candidate block positions is a large andprocessing time becomes huge when using the Cox et al. method.

As explained above, a method for detecting electronic watermarks onlocal distorted images is not believed to be provided by known systems.

SUMMARY OF THE INVENTION

According to embodiments of the present invention, a method ofdetermining a correspondence between patches in an original image havingn patches, and patches in a distorted image that constitutes theoriginal image distorted in accordance with local distortion, isprovided. The method involves inputting predetermining affine parametersof one of the plurality of predetermined patches; setting the inputpredetermining affine parameters of the one of the plurality of patch asinitial affine parameters for investigating a new patch; andinvestigating a correspondence of the new patch of the distorted imageand the new patch of the original image by slightly modifying the setinput predetermining affine parameters.

The invention also provides an embodiment of a method of determining acorrespondence between patches in an original image having n patches,and patches in a distorted image that constitutes the original imagedistorted in accordance with local distortion. Embodiments of thismethod involve inputting predetermining affine parameters of an m-thpatch for determining an (m+1)-th patch, wherein 1<m<n; setting theinput predetermining affine parameters of the m-th patch as initialaffine parameters for investigating the (m+1)-th patch; andinvestigating a correspondence of the (m+1)-th patch of the distortedimage and the (m+1)-th patch of the original image by slightly modifyingthe input predetermining affine parameters.

The invention further provides an embodiment of a method of detecting anelectronic watermark in a distorted image that constitutes a distortionof an original image in which the electronic watermark is embedded. Thismethod involves inputting the distorted image and comparisoninformation, the comparison information including at least one of theoriginal image or information used for embedding the electronicwatermark; dividing a domain of the original image into a plurality ofpatches, based on the comparison information; inputting affineparameters of a predetermined patch from among the patches in theoriginal image; on the basis of the affine parameters, extracting apatch candidate from the distorted image; using a predeterminedelectronic watermark detection method, judging whether the patchcandidate in the distorted image adequately correlates with aneighboring patch in the original image that neighbors the predeterminedpatch; and when the judging indicates an adequate correlation,outputting a part of the electronic watermark obtained by thepredetermined electronic watermark detection method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an original image I distorted to an image I′ in response toglobal distortion;

FIG. 2 shows an original image I distorted to an image I′ in response tolocal distortion;

FIG. 3 is a view of explaining a patch p in an image I by provided localdistortion;

FIG. 4 is a view of the relation between a patch pi and a patch pj;

FIG. 5 is a system block diagram of an embodiment of the presentinvention;

FIG. 6 is a functional block diagram of an electronic watermark detector10 and watermark extractor 11 (FIG. 5); and

FIG. 7 is a flow chart showing an example of a processing sequence forthe electronic watermark detector 10 (FIG. 5).

DETAILED DESCRIPTION OF THE EMBODIMENTS

With reference to the accompanying drawings, the embodiments of thepresent invention are explained in detail hereafter. The principle ofthe embodiments of the present invention is first explained as follows.

In local distortion, since vectors of neighboring pixels are almost thesame, neighboring pixels can be treated as a patch . A distorted imagedisplaced by local distortion can be expressed by two functions, such asformula (2). Therefore, investigating correspondence relations(correlation) between pixels in an original image and pixels in adistorted image is difficult, because it is necessary to investigate allpossible correspondence relationships.

Extreme distortion greatly spoils an impression of the original imageand reduces its value. To keep the impression of the original image, theoriginal is deformed little.

When an image is distorted for the purpose of altering or destroying anelectronic watermark, the patch of the distorted image can be regardedas a linear transformation. As shown FIG. 3, a patch p in an originalimage I displaced by local distortion can approximate a patch p′ in adistorted image I′ displaced by global distortion. The approximation isequivalent to the linear approximation of the two functions f and g.That is, it is expressed with the following formulas when two functionsare approximated around a certain point (x, y) in an original image.f(x+ε _(x) ,y+ε _(y))≈f(x,y)+ε_(x) f _(x)(x,y)+ε_(y) f _(y)(x,y),g(x+ε _(x) ,y+ε _(y))≈g(x,y)+ε_(x) g _(x)(x,y)+ε_(y) g _(y)(x,y)  (3)where:

-   a₁₁=f_(x)(x,y), a₁₂=f_(y)(x,y),-   a₂₁=g_(x)(x,y), a₂₂=g_(y)(x,y),-   b₁=f(x,y), b₂=g(x,y)    Then, formula (3) can be expressed as a formula of an affine    transformation.

Next, it is defined that image I is equivalent to sum of a patch p1 anda patch pn. As shown in FIG. 4, a patch p1 and a neighboring patch pjare deformed by local distortion. Each patch pi and pj has affineparameters of equation (4) by linear approximation around each startingpoint:a ^((i)) _(μv) ,b ^((i)) _(μ)(i=1, . . . ,n,μ=1,2,v=1,2)  (4)It can express by formula (5) between two patches pi and pj.a ^((i)) _(μv) =a ^((j)) _(μv)+δ^((i,j)) _(μv),b ^((i)) _(μ) =b ^((j)) _(μ)+Δ^((i,j)) _(μ)+δ^((i,j)) _(μ)  (5)Here, $\begin{matrix}{\begin{pmatrix}\Delta_{1}^{({i,j})} \\\Delta_{2}^{({i,j})}\end{pmatrix} = {\begin{pmatrix}a_{00}^{(j)} & a_{01}^{(j)} \\a_{10}^{(j)} & a_{11}^{(j)}\end{pmatrix}\begin{pmatrix}{\Delta\quad x^{({i,j})}} \\{\Delta\quad y^{({i,j})}}\end{pmatrix}}} & (6)\end{matrix}$where Δx^((i,j)),Δy^((i,j)) are gaps between each two patches of eachoriginal image.

The mapping of the affine parameters of the patch pi and the patch pj isexpressed with six parameters of equation (7):δ^((i,j)) _(μv),δ^((i,j)) _(μ)  (7)Since these parameters are near zero, it can efficiently determine theoptimum parameters by preferentially searching near zero.

As explained above, an original image is divided into patches. Adistorted image is equivalent to a sum of patches. A patch of thedistorted image, expressed using affine parameters, may be correlated toa patch in the original image. Patches neighboring a given patch may besought by slightly modifying the given patch's affine parameters as acenter point or starting point. For example, if the given patch's affineparameters are (1,1,1,1,1,1), then neighboring patches may be sought byusing slightly-modified affine parameters such as (2,1,1,1,1,1),(1,2,1,1,1,1), (1,1,2,1,1,1), (1,1,1,2,1,1), and so forth.

By using the above approach, searches of digital watermarks may be madeat higher speeds. On the basis of the above principles, the embodimentsof the invention are explained as below.

FIG. 5 shows a computer system implementing an exemplary embodiment.

The computer system 1, such as a personal computer or a server computer,comprises an input unit 2 such as a mouse and a keyboard, a display unit3, a control unit 4 comprising a microprocessor for controlling thewhole computer and for processing software programs such as anelectronic watermark detector 10 and electronic watermark extractor 11,etc., a communication unit 5 for communicating to external devices (notshown), and a storage unit 6 such as a semiconductor memory unit, harddisk drive unit, or CD/DVD drive unit. Storage unit 6 stores anoperating system program as well as software programs including:

-   -   electronic watermark detector 10,    -   electronic watermark extractor 11,    -   comparison information 12 used for a detection of digital        watermarks,    -   a detected image 13 in which is embedded an electronic        watermark,    -   various settings 14 used by electronic watermark detector 10 and        electronic watermark extractor 11, and    -   other programs or information as may be needed for a particular        embodiment.        Comparison information 12 may include one or more of:    -   an original image in which is embedded an electronic watermark,    -   an image of an electronic watermark,    -   information indicating a domain of an original image,    -   a random number sequence used for embedding the electronic        watermark in the original image, and    -   other information as may be needed for a particular embodiment.

FIG. 6 is a functional block diagram of the electronic watermarkdetector 10 and the electronic watermark extractor 11 according to anembodiment.

Electronic watermark detector 10 includes an affine parameter settingsection 21, a parameter generator 22, and a local watermark detector 23.Electronic watermark detector 10 may sometimes be referred to as a“patch” electronic watermark detector to distinguish it from “local”watermark detector 23.

Electronic watermark detector 10 inputs a detected image 13, comparisoninformation 12, as well as setting information 14. Setting information14 may included, for example:

-   -   patch size,    -   patch position,    -   affine parameters of the patch, and    -   other information as may be needed for a particular embodiment.        Electronic watermark detector 10 then outputs to electronic        watermark extractor 11, a result of detecting part of the        electronic watermark embedded in the detected image.

The electronic watermark detector 10 divides a domain of the originalimage into a plurality of patches, with the setting information 14, theinput patch size and comparison information 12. In order to extract eachpatch of the detected image 13 corresponding to each the dividedpatches, it judges by detecting the part of the electronic watermark inone of several patch candidates that are sequentially generated on thebasis of the detected image 13. Patch candidates are generated by usingthe formula of an affine transformation, using affine parameters of apreviously-determined neighboring patch as a starting point. Whether thegenerated patch candidate includes part of the electronic watermark isjudged. If part of the electronic watermark is included in the patchcandidate, the patch candidate is determined as a patch. If part of theelectronic watermark is not included in the patch candidate, a new patchcandidate is generated by changing the affine parameters a little. Byrepeating above process for all patches p1 to pn, the electronicwatermark of the detected image is detected.

The affine parameter setting section 21 sets up affine parameters for astart patch, and helps to detect a part of an electronic watermark laterin the process. If detecting the part of the electronic watermark from agiven patch of the detected image, the affine parameter setting section21 outputs the affine parameters of the given patch received fromsetting information 14. Later, when detecting part of the electronicwatermark from other patches, the affine parameter setting section 21outputs affine parameters received from the local watermark detector 23(see loopback path leading from local watermark detector 23 to affineparameter setting section 21 in FIG. 6). The affine parameters includethe six parameters explained previously.

The parameter generator 22 updates each the affine parameters input fromaffine parameter setting section 21 by a predetermined updatingtechnique, generates new affine parameters, and outputs thenewly-generated affine parameters to local watermark detector 23. In theaffine parameter update technique, whenever receiving a parameterrequest from the local watermark detector 23 on the basis of the affineparameters which set up from the affine parameter setting section 21, itpreferably changes parameters little by little.

Local watermark detector 23 extracts a patch candidate from the detectedimage 13 based on the affine parameters input from the parametergenerator 22. Local watermark detector 23 judges whether the patchcandidate should be extracted as a patch. When it should be extracted,the result detected from the patch is output to the electronic watermarkextractor 11, and the affine parameters at that time are output to theaffine parameter setting section 21.

The details of local watermark detector 23 are explained below.

Local watermark detector 23 inputs setting information 14 for specifyingeach patch, the detected image 13 and the comparison information 12, atthe time of the first stage. For example, the setting information 14comprise a patch size shown by vertical x side and position of eachpatch, in case of a rectangular patch. Also, local watermark detector 23calculates the formula of an affine transformation, and divides thedomain of the original image into a plurality of patches on the basis ofthe patch size.

The division into patches by local watermark detector 23 is based on thedomain of the image before, rather than after, local distortion. Thatis, the local watermark detector 23 extracts the domain of the originalimage before local distortion. Generally, the domain is a rectangle. Theextracted domain is substituted and changed into the formula of theaffine transformation using the affine parameters input from parametergenerator 22.

The part of the distorted image that is clipped by the changed domain isa patch candidate. The patch candidate is judged by a predeterminedmethod to be either true or false. If it is true only, the patchcandidate corresponds to a rectangular area of patch before localdistortion.

Methods of judging whether a patch candidate is true or false includethe following four exemplary methods.

1. First Judgment Method:

-   -   Each (rectangle) patch of the comparison information 12 is        obtained beforehand by dividing the comparison information 12        into the input patch size.    -   Each pixel value of the corresponding (rectangle) patch is        subtracted from each pixel value of the patch candidate of the        detected image.    -   If each pixel value of the subtracted result is close to zero,        then the patch candidate is judged “true;” otherwise it is        judged to be “false.”        When performing the first judgment method, comparison        information 12 preferably contains at least one of an original        image in which is embedded an electronic watermark, or an        electronic watermark image itself.

2. Second Judgment Method:

-   -   A domain of a patch after distortion is determined by using the        input setting information 14 and affine parameters of a detected        patch.    -   The pixel value of the pixel in the patch before distortion is        calculated from the pixel value of the pixel in the determined        domain after distortion.    -   A correlation value between the calculated pixel value and a        random number sequence used to embed an electronic watermark is        calculated.    -   If the correlation value is larger than the predetermined        threshold, the patch candidate is judged “true;” otherwise it is        judged “false.”        When performing the second judgment method, comparison        information 12 preferably contains the random number sequence        used to embed the electronic watermark.

3. Third judgment method: This method is applied when an electronicwatermark is embedded using frequency domain techniques.

-   -   The patch candidate after local distortion and the patch before        local distortion are changed into the frequency domain by, for        example, a Fourier transformation.    -   The difference between each frequency component of the patch        area and each frequency component of the patch candidate are        added (integrated).    -   If the result of adding (integration) is smaller than the        predetermined threshold, then the patch candidate is judged        “true;” otherwise it is judged “false.”        When performing the third method, comparison information 12        preferably contains either the original image or the electronic        watermark image.

4. Fourth judgment method: Like the third method, the fourth method isapplied when an electronic watermark is embedded using frequency domaintechniques.

-   -   The value of the each pixel in the patch before distortion is        calculated from the value of each pixel of the patch candidate,        and the obtained pixel values are changed into the frequency        domain by, for example, Fourier transformation.    -   A correlation value between a value of the changed frequency        component and a random number sequence used at the time of        embedding of an electronic watermark is calculated.    -   If the correlation value is larger than the predetermined        threshold, then the patch candidate is judged “true;” otherwise        it is judged “false.”        When performing the fourth judgment method, the comparison        information 12 preferably contains the random number sequence        used at the time of embedding of the electronic watermark.

Of course, the invention contemplates use of judgment methods, otherthan the four methods explained above.

When judged true by the foregoing methods, the patch candidate is judgedto correspond to domain local distortion, and simultaneously, it also isjudged that a digital watermark is embedded. On the other hand, whenjudged false, it means that the patch candidate does not correspond todomain local distortion.

In order to extract a new patch candidate, local watermark detector 23cancels a previous patch candidate and requires generation of newparameters by parameter generator 22. Thereby, the parameter generator22 generates new affine parameters, and outputs the generated affineparameters to local watermark detector 23. By repeating the aboveprocesses, the right patch of a detected image corresponding to a patchbefore local distortion is extracted.

In addition, in an actual system, the maximum number of times theprocess should be repeated in a search of one patch is predetermined.When a patch is detected before the maximum number of times, as thepatch candidate is output as a patch to the electronic watermarkextractor 11, affine parameters from the corresponding patch are outputto the affine parameter setting section 21.

On the other hand, even if the number of times reaches the maximum, whena patch is undetectable, it considers that the patch candidate generatedwith the first affine parameters to be a patch, and is output as a patchto the electronic watermark extractor 11, and the first affineparameters are output to the affine parameter setting section 21.Alternatively, it may consider that the patch candidate by whom thelargest (integration) result or correlation value were acquired is apatch, and the affine parameters may be output.

Furthermore, an indication that an electronic watermark has not beendetected is output to the electronic watermark extractor 11. The outputaffine parameters are used as start parameters at the time of anextraction of a following and neighboring patch.

The electronic watermark extractor 11 stores the position information onthe patch and whether the patch has a part of the electronic watermark,serially sent from the patch electronic watermark detector 10.

The electronic watermark extractor 11 extracts electronic watermarkafter all electronic watermarks on the detected image are detected.

FIG. 7 shows a flow chart of an exemplary process executed by the patchelectronic watermark detector 10. For purposes of this discussion of anexample, it is assumed that the first patch candidate judgment method,described above, is used.

First, initial information is received (S101). The initial informationincludes the comparison information 12, the detected image 13, affineparameters of a patch, patch size, and so on. The patch size may becontained in the setting information 14, may be specified directly froman input unit 2 such as a keyboard, or may be determined automaticallyby indicating a number of partitions of the patch by a user. The affineparameters of the patch may be contained in the setting information 14,or may be input from the input unit 2 depending on a user's choice, ormay be automatically initialized to “0” or some other suitable initialvalue.

Next, based on the input initial information, the domain of the originalimage obtained from the comparison information 12 is divided intopatches. A serial number (index) q is given to each patch, beginning at“1” (S103) and incremented in ascending order (S123, described below).

The domain of the original image may be the original image itself, or itmay be the electronic watermark image, or it may be a virtual domainthat follows the form of an original image.

Next, it takes coordinates that indicate a domain of the originalimage's patch that is specified by counter value q (S104). Also, pixelsof the original image patch specified by counter value q are extractedfrom the comparison information 12 (S105).

Meanwhile, the input affine parameters of the patch are held with theaffine parameter setting section 21 (S111). And initial value p=1 is setto a counter (not shown) that shows the number of times P that theaffine parameters change (S112).

Next, the held affine parameters are set to the formula of an affinetransformation (S113). A domain changed by substituting the position ofthe domain of the patch extracted at step S104 using the formula of theaffine transformation that set this affine parameter is obtained (S114),and pixels of a patch candidate are extracted from the domain obtainedin the detected image.

Next, the pixels in the patch of the original image obtained at stepS105 are compared (S115) to the pixels of the patch candidate obtainedat the step S114, by the above first judgment method or other suitablemethod.

When judged “false” as a result of the comparison, it judges (S116)whether the counter has reached a value “m.” Here, “m” is an upper limitof the number of times of repetition of extracting one patch from apatch candidate; when an adequately correlated patch is not found forwhatever reason, the upper limit prevents searching infinitely. Thevalue of “m” may be fixed internally, or may be specified at the time ofinitial information input (S101).

If p is not equal to M at step S116, counter value p is increased(S117), which increments the number p of affine parameter changes. Theaffine parameters are changed (S118) by only a small amount at one time,as explained with reference to FIG. 4. The changed affine parameters areinvolved in the performance of steps S113 and following.

When a patch is detected at step S115, or when a affine change countervalue p reaches the limiting value at step S116, the detection result ofthe electronic watermark of the patch is output to the electronicwatermark extractor 11 (S121). In the case of the former, the detectedwatermark is made into a detection result, and in the case of thelatter, it should just output a patch with an indication that awatermark was not detected therein.

The process ends (S125) if it is judged (q=n in S122) that theextraction of all n patches was completed after a detection resultoutput and all patches are detected.

If the extraction of all patches is not yet completed (q<n in S122), thepatch number counter value q is incremented (S123), a neighboring patchis specified, and control passes back to step S104. Also, the affineparameters of the extracted patch are output (S124) as a starting pointto search the neighboring patch, and control passes back to step S111.Since it is expected that affine parameters between neighboring patchesare changed only a little each time (as explained with reference to FIG.4), it is very effective to use these parameters as a starting point inthe next iteration.

Although the above is an explanation of the processes of the patchelectronic watermark detector 10 using the first judgment method, forexample, when the second judgment method is used, “a random numbersequence used for embedding the patch q is extracted” may instead of thestep S105, “the correlation value between each area is calculated and itcompares with threshold T” may instead of the step S115. The otherssteps may be same.

Also, when the third judgment method is used, “changing each area into afrequency domain, calculating the sum value of the difference, andcomparing with threshold T (it differing from the above-mentioned T)”may substitute for step S115. The others steps may be same.

In addition, when still other judgment methods are used, the changedpart according to the judgment technique does not deviate from the flowchart of FIG. 5. The same basic process is used.

Thus, the invention provides various embodiments.

According to embodiments of the present invention, a method ofdetermining a correspondence between patches in an original image havingn patches, and patches in a distorted image that constitutes theoriginal image distorted in accordance with local distortion, isprovided. The method involves inputting predetermining affine parametersof one of the plurality of predetermined patches; setting the inputpredetermining affine parameters of the one of the plurality of patch asinitial affine parameters for investigating a new patch; andinvestigating a correspondence of the new patch of the distorted imageand the new patch of the original image by slightly modifying the setinput predetermining affine parameters.

The invention also provides an embodiment of a method of determining acorrespondence between patches in an original image having n patches,and patches in a distorted image that constitutes the original imagedistorted in accordance with local distortion. Embodiments of thismethod involve inputting predetermining affine parameters of an m-thpatch for determining an (m+1)-th patch, wherein 1<m<n; setting theinput predetermining affine parameters of the m-th patch as initialaffine parameters for investigating the (m+1)-th patch; andinvestigating a correspondence of the (m+1)-th patch of the distortedimage and the (m+1)-th patch of the original image by slightly modifyingthe input predetermining affine parameters.

The invention further provides an embodiment of a method of detecting anelectronic watermark in a distorted image that constitutes a distortionof an original image in which the electronic watermark is embedded. Themethod involves inputting the distorted image and comparisoninformation, the comparison information including at least one of theoriginal image or information used for embedding the electronicwatermark; dividing a domain of the original image into a plurality ofpatches, based on the comparison information; inputting affineparameters of a predetermined patch from among the patches in theoriginal image; on the basis of the affine parameters, extracting apatch candidate from the distorted image; using a predeterminedelectronic watermark detection method, judging whether the patchcandidate in the distorted image adequately correlates with aneighboring patch in the original image that neighbors the predeterminedpatch; and when the judging indicates an adequate correlation,outputting a part of the electronic watermark obtained by thepredetermined electronic watermark detection method.

The method can further involve updating the affine parameters to newaffine parameters when the judging does not indicate an adequatecorrelation;, and possibly also extracting a new patch candidate fromthe distorted image, on the basis of the new affine parameters, andusing the predetermined electronic watermark detection method, judgingwhether the new patch candidate from the distorted image adequatelycorrelates to the neighboring patch in the original image.

The method can further involve storing the affine parameters as initialaffine parameters for use in extracting new patch candidates when thejudging indicates an adequate correlation.

The judging can involve subtracting pixels of the original image's patchfrom respective pixels of the distorted image's patch candidate; and ifa difference of the pixel subtracting is smaller than a predeterminedthreshold, judging the adequate correlation to exist.

Alternatively, the judging can involve calculating pixel values ofpixels in the patch before distortion, based on pixel values of a pixelin a determined domain in the distorted image; calculating correlationvalues between the calculated pixel values and a random number sequenceused to embed the electronic watermark; and if the correlation valuesare larger than a predetermined threshold, judging the adequatecorrelation to exist.

Alternatively, the judging can involve transforming into the frequencydomain, the patch candidate in the distorted image and the patch in theoriginal image so as to produce patch candidate frequency components andoriginal image patch frequency components; accumulating differencesbetween respective patch candidate frequency components and originalpatch frequency components; and if a result of the accumulating ofdifferences is smaller than a predetermined threshold, judging theadequate correlation to exist.

Alternatively, the judging can involve calculating values of pixels inthe patch in the original image based on values of pixels of the patchcandidate in the distorted image; transforming into the frequencydomain, the calculated values of the pixels in the patch in the originalimage, so as to produce original image patch frequency components;calculating a correlation value between the original image patchfrequency components and a random number sequence used to embed theelectronic watermark; and if the correlation value is larger than apredetermined threshold, judging the adequate correlation to exist.

According to the above embodiments, electronic watermarks can beefficiently detected from detected images even if they have beensubjected to local distortion. Moreover, the correspondence relationbetween sufficiently small domains can be easily found, both before andafter local distortion. Then, since the processes of electronicwatermark detection may be very high speed, application of the inventivemethod to actual products makes them very efficient.

According to the electronic watermark method and system explained above,even if local distortion is given to a picture image, the affinetransformation that approximates the deformation locally can be foundefficiently, and an electronic watermark can be restored.

Although the foregoing explanation focuses on still images, it is alsoenvisioned to apply the embodiments to moving images. In that case, thepatch may be in the same frame, or may be a patch in a former frame. Ifthe patch is in a former frame, the embodiments of the invention may beused by considering that patches may be neighbors in time rather thanmerely neighbors in space.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention is not limited to thespecific details and representative embodiments shown and describedherein. Accordingly, various modifications may be made without departingfrom the spirit or scope of the general inventive concept as defined bythe appended claims and their equivalents.

1-16. (Canceled)
 17. A method of determining a correspondence between patches in an original image having n patches, and patches in a distorted image that constitutes the original image distorted in accordance with local distortion, the method comprising steps of: a) inputting predetermined affine parameters of an m-th patch for determining an (m+1)-th patch, wherein 1<m<n; b) setting the input predetermined affine parameters of the m-th patch as initial affine parameters for investigating the (m+1)-th patch; and c) investigating a correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image by slightly modifying the input predetermined affine parameters.
 18. A method of determining a correspondence between patches in an original image having n patches, and patches in a distorted image that constitutes the original image distorted in accordance with local distortion, the method comprising steps of: a) inputting predetermined affine parameters of one of the plurality of predetermined patches; b) setting the input predetermined affine parameters of the one of the plurality of patch as initial affine parameters for investigating a new patch; and c) investigating a correspondence of the new patch of the distorted image and the new patch of the original image by slightly modifying the set input predetermined affine parameters.
 19. The method of claim 17, further comprising steps of: determining that the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria.
 20. The method of claim 17, wherein the investigating step c) includes sub-steps of: subtracting pixels of the original image (m+1)-th patch from respective pixels of the distorted image (m+1)-th patch; and if a difference of the pixel subtracting is smaller than a predetermined threshold, determining that the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria.
 21. The method of claim 17, wherein the investigating step c) includes sub-steps of: transforming into the frequency domain, the (m+1)-th patch of the original image and the distorted image (m+1)-th patch so as to produce distorted image (m+1)-th patch frequency components and (m+1)-th patch of the original image frequency components; accumulating differences between respective distorted image (m+1)-th patch frequency components and (m+1)-th patch of the original image frequency components; and if a result of the accumulating of differences is smaller than a predetermined threshold, determining that the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria.
 22. The method of claim 18, further comprising steps of: determining that the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria.
 23. The method of claim 18, wherein the investigating step c) includes sub-steps of: subtracting pixels of the new patch of the original image from respective pixels of the new patch of the distorted image; and if a difference of the pixel subtracting is smaller than a predetermined threshold, determining that the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria.
 24. The method of claim 18, wherein the investigating step c) includes sub-steps of: transforming into the frequency domain, the new patch of the original image and the new patch of the distorted image so as to produce new patch of the distorted image frequency components and new patch of the original image frequency components; accumulating differences between respective new patch of the distorted image frequency components and the new patch of the original image frequency components; and if a result of the accumulating of differences is smaller than a predetermined threshold, determining that the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria.
 25. An apparatus configured to determine a correspondence between patches in an original image having n patches, and patches in a distorted image that constitutes the original image distorted in accordance with local distortion, comprising: an input portion configured to input predetermined affine parameters of an m-th patch for determining an (m+1)-th patch, wherein 1<m<n; an initial affine parameter setting portion configured to set the input predetermined affine parameters of the m-th patch as initial affine parameters for investigating the (m+1)-th patch; and an investigating portion configured to investigate a correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image by slightly modifying the input predetermined affine parameters.
 26. The apparatus according to claim 25, further comprising: a determining portion configured to determine if the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria.
 27. The apparatus according to claim 25, wherein the investigating portion includes, a pixel subtraction portion configured to subtract pixels of the original image (m+1)-th patch from respective pixels of the distorted image (m+1)-th patch, and a determining portion configured to determine when the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria when a difference of the pixel subtracting is smaller than a predetermined threshold.
 28. The apparatus according to claim 25, wherein the investigating portion includes, a frequency domain transform portion configured to transform the (m+1)-th patch of the original image and the distorted image (m+1)-th patch into the frequency domain so as to produce distorted image (m+1)-th patch frequency components and (m+1)-th patch of the original image frequency components, an accumulation portion configured to accumulate differences between respective distorted image (m+1)-th patch frequency components and (m+1)-th patch of the original image frequency components, and a determining portion configured to determine that the correspondence between the (m+1)-th patch of the distorted image and the (m+1)-th patch of the original image meets pre-established correlation criteria when a result of the accumulating of differences is smaller than a predetermined threshold.
 29. An apparatus configured to determine a correspondence between patches in an original image having n patches, and patches in a distorted image that constitutes the original image distorted in accordance with local distortion, comprising: an input portion configured to input predetermined affine parameters of one of the plurality of predetermined patches; an initial affine parameter setting portion configured to set the input predetermined affine parameters of the one of the plurality of patch as initial affine parameters for investigating a new patch; and an investigating portion configured to investigate a correspondence of the new patch of the distorted image and the new patch of the original image by slightly modifying the set input predetermined affine parameters.
 30. The apparatus according to claim 29, further comprising: a determining portion configured to determine if the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria.
 31. The apparatus according to claim 29, wherein the investigating portion includes, a pixel subtraction portion configured to subtract pixels of the new patch of the original image from respective pixels of the new patch of the distorted image; and a determining portion configured to determine that the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria when a difference of the pixel subtracting is smaller than a predetermined threshold.
 32. The apparatus according to claim 25, wherein the investigating portion includes, a frequency domain transform portion configured to transform the new patch of the original image and the new patch of the distorted image into the frequency domain so as to produce new patch of the distorted image frequency components and new patch of the original image frequency components, an accumulation portion configured to accumulate differences between the new patch of the distorted image frequency components and the new patch of the original image frequency components, and a determining portion configured to determine that the correspondence between the new patch of the distorted image and the new patch of the original image meets pre-established correlation criteria when a result of the accumulating of differences is smaller than a predetermined threshold. 